Field of the Invention
Embodiments of the present invention relate to color, multi-spectral and hyper-spectral imaging systems such as still cameras, video cameras, scanners, microscopes and endoscopes and also to image and video compression systems.
Background Information
Studies of the human retina have shown that while L, M cones, corresponding to red, green are large in number, S cones, corresponding to blue, are very few and semi-regularly arranged. Furthermore it has been shown that small blue spots are not reliably seen by the Human Visual System (HVS) and appear to vanish under certain circumstances.
The so called methods of colorization have been developed in recent years that add color to monochrome images from a few pixels that do have color information. The original aim of these techniques was to aid artists in adding color to old black and white photos. (see A. Levin, D. Lischinski and Y. Weiss: Colorization Using Optimization, ACM Transactions on Graphics, vol. 23, pp. 689694, August 2004), (see G. Sapiro: Inpainting the colors, IMA Preprint Series 1979, Institute for Mathematics and Its Applications, University of Minnesota, May 2004).
A new class of image and video compression has been subsequently developed where the encoder compress luminance information conventionally and provide color information only on a selected few Representative Pixels. The decoder decompresses the luminance image and then colorizes it using color information in the Representative Pixels (see L. Cheng and S. V. N. Vishwanathan: Learning to Compress Images and Videos, Proceedings of 24th International Conference on Machine Learning (ICML), Vol. 227, pp. 161168, 2007), (see X. He, M. Ji, and H. Bao: A Unified Active and Semi-supervised Learning Framework for Image Compression, IEEE CVPR2009, pp. 6572, June 2009), (see T. Miyata, Y. Komiyama, and Y. Inazumi, Y. Sakai: Novel Inverse Colorization for Image Compression, Proceedings of Picture Coding Symposium, 2009).
Image sensors are overlaid with color filter arrays (CFA) to enable them to capture color images. A popular CFA is the Bayer pattern comprising of red, green and blue colors (see B. E. Bayer, Color imaging array, Jul. 20, 1976. U.S. Pat. No. 3,971,065).
Recently several RGBW CFAs have been proposed that contain White or transparent color filters in addition to red, green and blue (see J. T. Compton and J. F. Hamilton Jr., Image sensor with improved light sensitivity, U.S. Patent 20070024931A1, 2005). Images captured by RGBW sensors have suffered from color aliasing since each color is very sparsely sampled. This is because RGBW CFAs try to capture 4 colors as opposed to 3 colors for the successful Bayer CFA.
Recently multispectral and hyperspectral cameras have been proposed with a single sensor (see Y. Monno, M. Tanaka, and M. Okutomi, Multispectral demosaicking using adaptive kernel upsampling, Proc. of IEEE Int. Conf. on Image Processing, pp. 32183221, 2011), (see Y. Monno, M. Tanaka, and M. Okutomi, Multispectral demosaicking using guided filter, Proc. of SPIE, vol. 8299, pp. 82990O182990O7, 2012), (see L. Miao, H. Qi, R. Ramanath, and W. E. Snyder, Binary tree-based generic demosaicking algorithm for multispectral filter arrays, IEEE Trans. on Image Processing, vol. 15, no. 11, pp. 35503558, 2006).